Reflective Oracles: A Foundation for Game Theory in Artificial Intelligence

Classical game theory treats players as special: A description of a game contains a full, explicit enumeration of all players. This isn’t a realistic assumption for autonomous intelligent agents. Ideally, such agents would be based on a decision-theoretic foundation for game theory in which their coplayers are a non-distinguished part of the agent’s environment, but it is nontrivial to find such a foundation. Attempts to model both players and the environment as Turing machines, for example, fail for standard diagonalization reasons. In this paper, we consider oracle machines with access to a “reflective” oracle, which is able to answer questions about the outputs of other machines with access to the same oracle. These oracles avoid diagonalization by answering some queries randomly. We show that machines with access to a reflective oracle can be used to define rational agents using causal decision theory. These agents model their environment as a probabilistic oracle machine, which may contain other agents as a non-distinguished part. We show that if such agents interact, they will play a Nash equilibrium, with the randomization in mixed strategies coming from the randomization in the oracle’s answers. This can be seen as providing a foundation for classical game theory in which players aren’t special.